Skip to content

Functions for flushing done tasks and evicted objects.#2033

Merged
pcmoritz merged 1 commit intoray-project:masterfrom
robertnishihara:moreflushing
May 18, 2018
Merged

Functions for flushing done tasks and evicted objects.#2033
pcmoritz merged 1 commit intoray-project:masterfrom
robertnishihara:moreflushing

Conversation

@robertnishihara
Copy link
Collaborator

@robertnishihara robertnishihara commented May 10, 2018 •
edited
Loading

This lets you flush metadata from Redis for done tasks and evicted objects.

ray.experimental.flush_done_tasks_unsafe()
ray.experimental.flush_evicted_objects_unsafe()

If these functions are too slow, they can be called from within a remote function running in the background.

You can also flush each shard individually, for example

@ray.remote
def flush_task_shard(i):
    ray.experimental._flush_finished_tasks_unsafe_shard(i)

@ray.remote
def flush_object_shard(i):
    ray.experimental._flush_evicted_objects_unsafe_shard(i)

task_flushing = [flush_task_shard.remote(i) for i in range(len(ray.global_state.redis_clients))]
object_flushing = [flush_object_shard.remote(i) for i in range(len(ray.global_state.redis_clients))]
%time ray.get(task_flushing + object_flushing)

The latter should get faster as the number of Redis shards is increased.

cc @TWAnthony

@AmplabJenkins
Copy link

AmplabJenkins commented May 10, 2018

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/5316/
Test FAILed.

@AmplabJenkins
Copy link

AmplabJenkins commented May 10, 2018

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/5319/
Test FAILed.

@AmplabJenkins
Copy link

AmplabJenkins commented May 10, 2018

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/5320/
Test PASSed.

@AmplabJenkins
Copy link

AmplabJenkins commented May 11, 2018

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/5325/
Test PASSed.

@AmplabJenkins
Copy link

AmplabJenkins commented May 16, 2018

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/5408/
Test PASSed.

@AmplabJenkins
Copy link

AmplabJenkins commented May 17, 2018

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/5445/
Test PASSed.

@AmplabJenkins
Copy link

AmplabJenkins commented May 18, 2018

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/5464/
Test FAILed.

@pcmoritz pcmoritz merged commit 78e4b02 into ray-project:master May 18, 2018
@pcmoritz pcmoritz deleted the moreflushing branch May 18, 2018 09:00
alok added a commit to alok/ray that referenced this pull request May 18, 2018
@alok
Merge branch 'master' into comps
0285567 
* master: (22 commits)
  [xray] Fix bug in updating actor execution dependencies (ray-project#2064)
  [DataFrame] Refactor __delitem__ (ray-project#2080)
  [xray] Better error messaging when pulling from self. (ray-project#2068)
  Use source code in hash where possible (fix ray-project#2089) (ray-project#2090)
  Functions for flushing done tasks and evicted objects. (ray-project#2033)
  Fix compilation error for RAY_USE_NEW_GCS with latest clang. (ray-project#2086)
  [xray] Corrects Error Handling During Push and Pull. (ray-project#2059)
  [xray] Sophisticated task dependency management (ray-project#2035)
  Support calling positional arguments by keyword (fix ray-project#998) (ray-project#2081)
  [DataFrame] Improve performance of iteration methods (ray-project#2026)
  [DataFrame] Implement to_csv (ray-project#2014)
  [xray] Lineage cache only requests notifications about remote parent tasks (ray-project#2066)
  [rllib] Add magic methods for rollouts (ray-project#2024)
  [DataFrame] Allows DataFrame constructor to take in another DataFrame (ray-project#2072)
  Pin Pandas version for Travis to 0.22 (ray-project#2075)
  Fix python linting (ray-project#2076)
  [xray] Fix GCS table prefixes (ray-project#2065)
  Some tests for _submit API. (ray-project#2062)
  [rllib] Queue lib for python 2.7 (ray-project#2057)
  [autoscaler] Remove faulty assert that breaks during downscaling, pull configs from env (ray-project#2006)
  ...
alok added a commit to alok/ray that referenced this pull request May 21, 2018
@alok
Merge branch 'master' into fix-a3c-2
662eaa5 
* master: (24 commits)
  Performance fix (ray-project#2110)
  Use flake8-comprehensions (ray-project#1976)
  Improve error message printing and suppression. (ray-project#2104)
  [rllib] [doc] Broken link in ddpg doc
  YAPF, take 3 (ray-project#2098)
  [rllib] rename async -> _async (ray-project#2097)
  fix unused lambda capture (ray-project#2102)
  [xray] Use pubsub instead of timeout for ObjectManager Pull. (ray-project#2079)
  [DataFrame] Update _inherit_docstrings (ray-project#2085)
  [JavaWorker] Changes to the build system for support java worker (ray-project#2092)
  [xray] Fix bug in updating actor execution dependencies (ray-project#2064)
  [DataFrame] Refactor __delitem__ (ray-project#2080)
  [xray] Better error messaging when pulling from self. (ray-project#2068)
  Use source code in hash where possible (fix ray-project#2089) (ray-project#2090)
  Functions for flushing done tasks and evicted objects. (ray-project#2033)
  Fix compilation error for RAY_USE_NEW_GCS with latest clang. (ray-project#2086)
  [xray] Corrects Error Handling During Push and Pull. (ray-project#2059)
  [xray] Sophisticated task dependency management (ray-project#2035)
  Support calling positional arguments by keyword (fix ray-project#998) (ray-project#2081)
  [DataFrame] Improve performance of iteration methods (ray-project#2026)
  ...
alok added a commit to alok/ray that referenced this pull request May 24, 2018
@alok
Merge branch 'fix-a3c-torch' into trpo
9aac5bd 
* fix-a3c-torch: (37 commits)
  Add missing channel major
  Use correct filter size
  Add TODO
  Fix shape errors
  fmt
  Performance fix (ray-project#2110)
  Use flake8-comprehensions (ray-project#1976)
  Improve error message printing and suppression. (ray-project#2104)
  [rllib] [doc] Broken link in ddpg doc
  YAPF, take 3 (ray-project#2098)
  [rllib] rename async -> _async (ray-project#2097)
  fix unused lambda capture (ray-project#2102)
  [xray] Use pubsub instead of timeout for ObjectManager Pull. (ray-project#2079)
  [DataFrame] Update _inherit_docstrings (ray-project#2085)
  [JavaWorker] Changes to the build system for support java worker (ray-project#2092)
  [xray] Fix bug in updating actor execution dependencies (ray-project#2064)
  [DataFrame] Refactor __delitem__ (ray-project#2080)
  [xray] Better error messaging when pulling from self. (ray-project#2068)
  Use source code in hash where possible (fix ray-project#2089) (ray-project#2090)
  Functions for flushing done tasks and evicted objects. (ray-project#2033)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@concretevitamin concretevitamin concretevitamin approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@robertnishihara @AmplabJenkins @concretevitamin @pcmoritz